Section: Partnerships and Cooperations

International Initiatives

Hermes

• Title: Accelerating the Performance of Multi-Site Scientific applications through Coordinated Data management.

• International Partner (Institution - Laboratory - Researcher):

  • Lawrence Berkeley National Laboratory (United States) - Scientific Data Management Group - Suren Byna.

• Start year: 2019

• See also: http://hermes-ea2019.gforge.inria.fr.

• Advances in computing, experimental, and observational facilities are enabling scientists to generate and analyze unprecedented volumes of data. A critical challenge facing scientists in this era of data deluge is storing, moving, sharing, retrieving, and gaining insight from massive collections of data efficiently. Existing data management and I/O solutions on high-performance computing (HPC) systems require significant enhancements to handle the three V’s of Big Data (volume, velocity, and variety) in order to improve productivity of scientists. Even more challenging, many scientific Big Data and machine learning applications require data to be shared, exchanged, and transferred among multiple HPC sites. Towards overcoming these challenges, in this project, we aim at accelerating scientific Big Data application performance through coordinated data management that addresses performance limitations of managing data across multiple sites. In particular, we focus on challenges related to the management of data and metadata across sites, distributed burst buffers, and online data analysis across sites.

Huazhong university of Science and Technology (HUST):

We collaborate on resource management for stream data applications in the edge, I/O scheduling for SDDs and network-aware task scheduling for MapReduce.

National University of Singapore (NUS):

We collaborate on resource management for workflows in the clouds and optimizing graph processing in geo-distributed data-centers.

ShenZhen University:

We collaborate on resource management for workflows in the clouds and optimizing graph processing in geo-distributed data-centers.